Skip to main content
Version: v2.10.0

Troubleshooting

Most failed deployments fall into one of two categories, and distinguishing between them saves considerable time. Either CloudFormation refused the stack, in which case the message names the offending parameter or the missing permission and you're finished in a minute, or provisioning succeeded and the platform still isn't answering, which means the user data script died somewhere and the explanation is waiting in /var/log/bootstrap.log on the instance. The template sends no cfn-signal, so a green stack proves nothing whatsoever about the software.

Establish which category you're in first, then read the matching section.

Validation fails before any resource is created

A message like "RdsSubnet1Id must be provided when RdsCreate is true" comes from the template's own Rules section rather than from a resource. Two rules exist. RequireRdsSubnetsWhenEnabled demands RdsSubnet1Id, RdsSubnet2Id and RdsEngineVersion whenever RdsCreate is true, and RequireHttpsArtifacts demands CustomDnsName, HttpsCertS3Path and HttpsKeyS3Path whenever EnableHttps is true.

Fill in the named parameter, or turn the feature back off. Nothing has been created yet, so there's nothing to clean up.

Parameters do not exist in the template

The error reads Parameters: [AwsRegion, YeeduDockerUsername, YeeduDockerPassword] do not exist in the template. That's the shipped parameters.json.example being used unedited. Those three keys were removed from the template, and there are 19 parameters now. Delete the stale entries and rerun create-stack.

Requires capabilities: CAPABILITY_NAMED_IAM

The stack creates an IAM user and an IAM policy with names of its own choosing, so CloudFormation won't proceed without an explicit acknowledgement. Add --capabilities CAPABILITY_NAMED_IAM to the CLI call, or tick the box on the Configure stack options page in the console.

User is not authorized to perform some action

The deploying principal is missing a permission. CloudFormation acts entirely on its behalf, so the policy has to cover every service the template touches, which is a longer list than people expect: EC2, IAM, S3, EFS, RDS, ECR, CloudWatch Logs, Secrets Manager and CloudFormation itself. Compare what you've attached against the policy on the Prerequisites page. The ecr: and rds: actions only matter when the matching creation flags are on.

Resource already exists

This one has three flavours, and the retained-resource flavour is the one that catches people on a second deployment.

S3 bucket names are globally unique across all of AWS, and the bucket is always named ${AWS::StackName}-data. If someone else already holds that name, change the stack name.

The 17 ECR repositories and the 10 CloudWatch log groups are declared with DeletionPolicy: Retain, so deleting the stack leaves them behind by design. Recreating the stack in the same account and region then fails, because yeedu_reactive_actors or yeedu_userdata_logs is still sitting there. Set CreateContainerRepositories and CreateLogGroups to false and the stack will use the existing ones.

The AMI parameter is rejected

Ec2ImageId is validated against ^(?!.*\s)ami-[0-9a-fA-F]{8,}$ with a minimum length of 12, and the usual culprit is a trailing space picked up from a copy and paste. It also has to be an AMI in the region you're deploying to, since the type is AWS::EC2::Image::Id and AMI IDs are regional.

Stack is CREATE_COMPLETE but nothing responds

Log in and read the bootstrap log. This is where a stack that looks perfect turns out to have failed 30 seconds into user data.

sudo tail -200 /var/log/bootstrap.log
sudo docker ps

We see three causes far more often than any others. The first is no route to the internet from the subnet, which stalls the apt install and every download after it, since the template creates no NAT gateway and no VPC endpoints. The second is a failed sync from s3://yeedu-softwares/releases/, which leaves /opt/yeedu-software empty and stops the release from ever unpacking. The third is the ECR login, aws ecr get-login-password piped into docker login, failing because the instance profile couldn't reach the ECR endpoint.

If CreateLogGroups was true, the same output is in the yeedu_userdata_logs group in CloudWatch, shipped from /var/log/cloud-init-output.log by the agent.

No public IP on the instance

AssignPublicIp controls both the network interface property and the HasPublicIp condition that creates the Elastic IP. Setting it to true isn't enough on its own if the subnet doesn't support public addressing, so check that the subnet has public IP mapping enabled and a route to an internet gateway.

The InstancePublicIp and InstanceElasticIp outputs are conditional, so they simply won't appear when the parameter is false.

Stack deletion fails

CloudFormation can't delete a bucket that still has objects in it. Empty it and delete again.

aws s3 rm s3://yeedu-platform-data --recursive
aws cloudformation delete-stack --stack-name yeedu-platform

Anything else, check the events. The failure reason for the specific resource is usually the whole answer.

aws cloudformation describe-stack-events --stack-name yeedu-platform \
--query 'StackEvents[?ResourceStatus==`DELETE_FAILED`]'

Bear in mind that Secrets Manager keeps deleted secrets recoverable for a retention window, so a name like yeedu-platform-dev-yeedu-config-secret can block a redeployment until it's force-deleted or the window expires.